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1.  INTRODUCTION 


1.1  General  Overview  of  the  Human  Performance  Modeling  Effort 


The  Human  Performance  Model  (HPM)  was  developed  as  a  part  of  the  Infrared 
Tactical  Decision  Aid  Human  Performance  Modeling  and  Analysis  Program.  The  HPM 
effort  describes  an  approach  for  the  modification  of  the  sensor  performance  model 
component  of  Tactical  Decision  Aids  (TDA)  to  enable  the  accurate  prediction  of  human 
visual  detection,  identification  and  recognition  ranging  algorithms  for  various  target 
classes.  Additionally,  the  HPM  greatly  broadens  predictive  capability  to  any  arbitrary 
target  or  target  complex,  and  to  any  background  clutter  conditions. 

An  observer  model  is  incorporated  which  more  completely  utilizes  information  in  the 
detailed  thermal  image  generated  by  the  advanced  Thermal  Contrast  Model  (TCM2).  The 
general  approach  involves  taking  the  present  sensor  performance  model  and  subdividing  it 
into  independent  sensor,  display,  and  observer  models.  The  sensor  model  receives  as 
input  the  atmospherically  transformed  2D  target/background  image  produced  by  TCM2 
and  produces  a  2D  image  output  that  reflects  the  sensor  transformation/degradation  only. 
The  display  model  receives  as  input  the  sensor  output  image  and  similarly  applies  the 
display  transformation/degradation  to  simulate  the  actual  visual  stimulus  (cockpit 
display)  seen  by  a  human  pilot. 

The  HPM  transforms  the  output  of  the  display  module  (i.e.  the  predicted  cockpit 
display  image)  into  a  space/spatial  frequency  (S/SF)  representation.  This  representation  is 
a  multiple  spatial  frequency  channel  pyramid  representation  that  simulates  the  encoding 
used  by  the  human  visual  system.  The  representation  takes  the  form  of  a  4-D 
space/spatial  frequency  transform  using  gabor  basis  filters,  known  as  the  Hexagonal 
Orthogonal  Oriented  Pyramid.  The  image  processing  transformations  represent  modem 
vision  theory  and  are  based  on  current  psychophysical  and  neurophysiological 
experimentation  and  models.  The  representation  incorporates  both  luminance  gain  control 
(LGC)  and  contrast  gain  control  (CGC)  algorithms.  The  LGC  models  the  changes  in 
spatiotemporal  contrast  sensitivity  which  occur  as  a  fimction  of  ambient  illumination. 

The  CGC  models  the  changes  in  sensitivity  which  occur  locally  in  the  visual  system  due 
to  local  contrast  distribution  variations  in  a  scene  allowing  the  cortical  representation  for 
input  images  with  any  arbitrary  contrast  distribution. 

Predictions  about  observer  performance  are  then  obtained  by  applying  decision 
operators  in  a  simulated  pairwise  discrimination  task  to  the  population  of  mechanisms  in 
the  visual  encoding  representation.  The  paired  comparisons  task  is  amenable  to 
reformulation  for  the  accurate  prediction  of  a  vdde  range  of  visual  performance  tasks  such 
as  detection,  identification  and  recognition  ranges,  or,  in  more  complicated  tasks  such  as 
detecting  distributed  targets  in  cluttered  backgrounds. 
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1.2  Background 


The  Infrared  Taetical  Decision  Aid  (IRTDA)  was  developed  by  Air  Weather  Service 
as  an  analysis  tool  that  helps  a  commander  decide  how  to  deploy  forces  and  weapons  by 
providing  accurate  range  predictions  of  a  pilot’s  ability  to  detect,  identify,  and  recognize  a 
target  given  a  particular  set  of  weather  conditions  and  a  particular  sensor.  The  objective 
of  the  IRTDA  development  program  was  the  accurate  prediction  of  the  performance  of 
the  IR  weapons  system  including  both  precision  guided  munitions  (e.g.  I^R  Maverick)  and 
on-board  aircraft  target  acquisition  systems  (e.g.  LANTIRN).  The  existing  IRTDA 
consists  of  three  main  components:  a)  the  Thermal  Contrast  Model  (TCM2)  predicts  the 
target/background  scene,  b)  the  Atmospheric  Transmission  Model  (LOTRAN7)  models 
the  transmission  from  the  target  to  the  sensor,  and  c)  the  Sensor  Performance  Model 
(SPM)  then  predicts  sensor  performance  and  the  maximal  detection,  identification,  and 
recognition  (DIR)  ranges  of  the  human  observer. 

Over  the  years,  both  the  TCM2  and  LOTRAN7  have  evolved  into  refined 
computational  approaches  founded  in  verified  fundamental  physical  principles  and 
essentially  approach  first  principle  models.  As  such,  they  have  served  as  successful 
engineering  models  with  broad  application  to  problems  involving  infrared  (IR)  image 
prediction.  The  SPM  lumps  together  both  the  sensor  modeling  and  observer  modeling. 
The  sensor  portion  of  the  SPM  has  also  been  very  successful  in  its  predictive  abilities. 
However,  the  observer  portion  of  the  SPM  can  produce  ‘order  of  magnitude’  predictive 
error.  The  SPM  has  suffered  from  the  same  empirical  model  that  was  used  in  the  early 
versions  of  the  system,  which  is  based  upon  empirical  results  obtained  by  John  Johnson 
(Johnson,  1958).  Updating  the  human  observer  portion  of  the  SPM  has  failed  to  keep 
pace  with  advances  in  the  other  IRTDA  model  components. 

The  purpose  of  the  IRTDA  Human  Performance  Modeling  and  Analysis  Program 
was  to  correct  the  current  shortcomings  and  broaden  the  predictive  capability  of  the 
IRTDA  model  to  include  new  high  value  target  classes.  The  IRTDA  Human  Performance 
Modeling  and  Analysis  Program  has  demonstrated  why  the  current  SPM  is  insufficient  to 
characterize  the  performanee  of  an  observer  and  provides  a  solution  based  on  principles 
which  have  emerged  in  human  visual  science  over  the  last  two  decades.  This  effort 
resulted  in  the  Human  Performance  Model  (HPM).  The  sections  below  describe  the 
architecture  and  major  components  of  the  HPM  Human  Performance  Model,  the  system 
parameters  and  calibration,  and.^the  models’  applicability. 

The  HPM  offers  several  advantages  including  the  exploitation  of  all  target  facet 
information  from  TCM2,  and,  applicability  to  a  wide  range  of  targets  and  backgroimds 
with  complex  shapes  and  internal  structures.  Additionally,  the  approach  is  modular  and 
allows  for  independent  model  testing. 
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2.  OVERVIEW  OF  MODEL 


2.1  General  Description  of  the  Human  Performance  Model 

The  Human  Performance  Model  (HPM)  is  a  computational  model  of  human  visual 
discrimination.  That  is,  given  a  pair  of  input  images,  the  HPM  can  predict  whether  an 
average  human  observer  can  reliably  discriminate  between  those  two  images. 

Figure  1  summarizes  the  signal  flow  and  major  transformation  stages  in  the  HPM. 
First,  the  HPM  receives,  as  input,  a  pair  of  two-dimensional  achromatic  images,  denoted 
respectively  as  the  target  and  background  images.  Second,  each  image  is  transformed  into 
a  separate  four-dimensional  cortical  representation  that  serves  to  approximate  the 
distributed  neural  image  representation  found  in  the  primary  visual  cortex  of  humans 
when  viewing  that  same  image.  Third,  a  difference  vector  is  formed  by  subtracting  the 
cortical  representation  of  the  background  image  from  the  cortical  representation  of  the 
target  image.  Finally,  the  elements  in  the  difference  vector  are  pooled  in  a  nonlinear 
fashion  to  obtain  a  single  non-negative  sensitivity  value. 


Figure  1.  Overview  of  the  Human  Performance  Model. 


The  sensitivity  value  is  the  sole  output  of  the  HPM.  By  definition,  a  sensitivity 
value  of  0.0  is  obtained  from  identical  images,  a  value  of  1 .0  is  obtained  from  images 
whose  small  difference  places  them  near  the  threshold  for  visual  discrimination,  and  a 
value  much  larger  than  1.0  is  obtained  from  an  image  pair  that  is  highly  discriminable.  As 
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discussed  below,  the  output  of  the  Human  Performance  Model  (D)  can  be  converted  into 
a  predicted  probability  of  correct  discrimination. 


2.2  Computing  a  Cortical  Representation 


As  indicated  in  the  analysis  stage  of  Figure  1,  the  HPM  transforms  each  input  image 
into  a  four-dimensional  cortical  representation.  This  cortical  representation  serves  to 
approximate  the  distributed  neural  image  representation  found  in  the  primary  visual 
cortex  of  humans  when  viewing  each  image.  The  steps  required  to  compute  the  cortical 
representation  are  depicted  in  Figure  2  below.  The  three  major  transformation  stages  are 
the  luminance  gain  control,  the  space/spatial-frequency  transform,  and  the  contrast  gain 
control.  These  three  stages  are  briefly  discussed  below. 


Figure  2.  Block  Diagram  of  computing  a  cortical  representation. 


4 


2.2.1  Luminance  Gain  Control 


As  with  all  biological  neurons,  neurons  in  the  early  visual  system  have  a  very  limited 
dynamic  range  of  approximately  2  orders  of  magnitude  (10^).  Yet  these  neurons  respond 
in  a  graded  fashion  to  luminance  over  a  much  larger  range  spanning  about  10  orders  of 
magnitude  (10*®).  The  early  visual  system  accomplishes  this  feat  by  dynamically 
adjusting  its  sensitivity  in  accordance  with  the  prevailing  mean  luminance  of  the  imaged 
scene.  As  a  result,  the  spatial  contrast  sensitivity  of  the  human  visual  system  varies 
dramatically  as  a  function  of  mean  luminance.  These  changes  are  reflected  not  only  in  the 
scale,  but  also  the  shape,  of  human  sensitivity  to  spatial  sinewave  gratings  (Kelly,  1972). 
The  causes  of  these  luminance-related  sensitivity  changes  are  both  optical  and  neural. 
With  regard  to  optical  causes,  one  must  consider  the  effect  of  pupillary  size  adaptation. 
With  regard  to  neural  causes,  one  must  consider  the  effect  of  sensitivity  changes  and 
spatial  filtering  by  neurons  in  both  the  retina  and  the  lateral  geniculate  nucleus  (LGN). 
The  luminance  gain  control  module  models  the  overall  effect  of  the  known  mechanisms 
which  contribute  to  these  gain  changes.  As  shown  in  Figure  2,  the  luminance  gain  control 
is  implemented  as  a  set  of  luminance-dependent  spatial  filters.  The  shapes  and  gains  of 
these  spatial  filters  were  chosen  to  model  the  neurophysiological  data  from  higher 
primates  and  the  psychophysical  data  from  human  subjects. 


2.2.2  Space/Spatial  Frequency  Transform 


During  the  past  three  decades,  psychophysical  and  neurophysiological  experiments 
have  established  that,  for  each  subregion  in  visual  space,  there  is  a  set  of  neurons  in  the 
primary  visual  cortex  whose  members  are  each  selective  for  a  different  limited  region  in 
2-D  spatial  frequency.  This  discovery  precipitated  a  revolution  in  computational  visual 
science  leading  to  a  new  generation  of  models  of  visual  function  in  which  the  cortical 
representation  takes  the  form  of  a  4-D  space/spatial  frequency  wavelet  transform  (for  a 
review  see  Watson,  1990).  A  wavelet  transform  is  one  in  which  the  basis  functions  can 
all  be  made  equal  to  a  master  wavelet  by  scaling  or  rotation.  The  master  wavelet  in  visual 
cortex  models  is  generally  bandpass  and  oriented,  and  it  is  generally  complex-valued, 
having  real  and  imaginary  components  with  a  quadrature  phase  relationship.  The  HPM 
uses  a  transform,  known  as  the  Hexagonal  Orthogonal  Oriented  Pyramid  (HOP) 
transform,  based  on  a  gabor  basis  function  representation  that  satisfies  the  above 
properties  (Watson  and  Ahumada,  1987).  The  HOP  transform  is  a  complete,  orthogonal 
wavelet  transform.  During  this  project,  a  FAST  HOP  transform  algorithm  was  developed 
for  use  in  the  HPM.  As  indicated  in  Figure  2,  after  computing  the  HOP  transform,  the 
HPM  computes  a  squared  modulus  to  obtain  a  space/spatial  frequency  energy 
representation.  An  energy  representation  is  known  to  be  encoded  by  the  class  of  cells  in 
the  primary  visual  cortex  known  as  complex  cells  which  provide  the  major  input  to  higher 
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cortical  visual  areas  in  humans.  As  such,  this  neuronal  class  is  believed  to  be  critical  to 
visual  pattern  recognition  in  humans. 


2.2.3  Contrast  Gain  Control 


The  human  visual  system  locally  adapts  its  contrast  sensitivity  in  accordance  with 
the  local  retinal  contrast  distribution  provided  in  a  visual  scene.  Such  contrast  adaptation 
is  most  evident  in  psychophysical  experiments  that  involve  moderate  to  high  contrast 
stimuli.  Recently,  it  has  been  established  that  neurons  in  the  primary  visual  cortex 
typically  show  a  similar  adaptation.  Such  adaptation  by  cortical  neurons  is  modeled 
within  the  HPM  by  the  contrast  gain  control  module  illustrated  in  Figure  2.  Details  of  the 
adaptation  model  are  described  in  a  later  section  of  this  report.  The  parameters  that 
govern  the  contrast  gain  control  were  chosen  to  optimize  the  capability  of  the  HPM  to 
predict  published  contrast  sensitivity  data  obtained  in  human  psychophysical 
experiments. 


2.3  Multiple  Mechanism  Detector 


2.3.1  Theory 

In  order  to  relate  the  model  to  human  behavior  we  must  first  define  the  visual 
discrimination  task.  In  what  follows  we  assume  that  the  task  is  a  two-alternative  forced 
choice  task  in  which  the  observer  views  two  stimuli,  a  base  stimulus  and  a  base  +  test 
stimulus,  and  must  report  which  of  the  two  stimuli  contained  the  test.  Under  these 
conditions,  the  observer’s  psychometric  function  (probability  of  correct  discrimination  as 
a  function  of  stimulus  contrast)  will  vary  from  0.5  when  the  observer  is  simply  guessing 
to  1.0  for  perfect  discrimination  performance.  Quick  (1974)  has  shown  that  human 
psychometric  functions  are  well  described  by  the  following  equation: 

where  C  is  the  contrast  of  the  test  stimulus,  S  is  the  sensitivity  of  the  visual  system 
to  the  test  stimulus,  and  Q  is  a  variable  that  determines  the  steepness  of  the  psychometric 
function.  For  this  function  the  product  of  SC  is  equal  to  unity  when  the  probability  of 
correct  detection  is  0.75.  Therefore,  if  we  define  threshold  as  a  0.75  probability  of  correct 
detection,  then  S  is  simply  the  reciprocal  of  the  test  contrast  at  threshold. 

Examples  of  the  Quick  psychometric  function  are  shown  in  Figure  3.  Note  that 
when  test  contrast  is  equal  to  1/S  then  the  probability  of  correct  discrimination  is  0.75. 
Comparison  of  the  two  curves  on  the  right  shows  how  increasing  Q  increases  the 
steepness  of  the  psychometric  function. 
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Figure  3.  Quick  psychometric  function  for  a  two-alternative  forced  choice 
experiment. 


The  importance  of  the  Quick  formulation  is  that  it  provides  a  simple  way  to 
compute  the  overall  sensitivity  of  a  system  with  multiple  mechanisms.  Figure  4  shows 
tlie  human  performance  model  expressed  schematically  as  a  multichannel  model  with 
noise  added  prior  to  signal  detection. 
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Figure  4.  Multiple  Mechanism  Model  with  Noise  Perturbation. 

Quick  showed  that  the  overall  sensitivity  of  a  multichannel  system  with  individual 
channel  sensitivities  5,  is  given  by  the  following  equation: 


S  = 


’'thresh  L  i 


-ii/e 


where  S  is  the  Human  Performance  Model  sensitivity,  is  contrast  at  threshold  , 
Sj  is  the  sensitivity  of  the  channel  and  the  Quick  exponent,  Q,  is  the  steepness 
parameter  given  above.  Quick’s  original  formulation  was  limited  to  threshold  stimuli  and 
did  not  include  a  nonlinear  contrast  response  function  which  is  necessary  to  account  for 
suprathreshold  sensitivity.  However,  Wilson  (1980)  extended  Quick’s  formulation  to 
suprathreshold  systems  and  showed  that  the  overall  sensitivity  of  the  Human 
Performance  Model  shown  above  is  given  by: 
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s=- 


^thresh  L  / 


S|s, 


\PQ 


■\VPQ 


This  function  is  exactly  the  same  as  Quick’s  function  except  that  the  exponent  Q  has 
been  replaced  by  the  product  PQ  where  P  is  the  correlation  between  the  noise  processes 
added  to  each  mechanism.  Thus  PQ  will  only  equal  Q  when  P  =1 .0,  i.e.,  when  the  noise 
among  mechanisms  is  perfectly  correlated.  There  is  at  present  no  conclusive  evidence 
concerning  the  degree  to  which  the  noise  is  correlated  among  visual  channels. 
Psychophysical  experiments  can  only  measure  the  product  PQ  which  varies  from  roughly 
2.5  -  4.5  for  threshold  studies  and  increases  when  discriminating  high  contrast  stimuli. 
Because  we  cannot  determine  the  values  of  P  and  Q,  we  will  assume  for  simplicity  that 
the  noise  among  the  detectors  is  perfectly  correlated  (PQ  =  Q).  Note  that  it  is  the 
product  PQ  that  is  important  for  predicting  the  discrimination  behavior  of  human 
observers.  Thus  while  the  noise  correlation  among  channels  is  important  from  a 
theoretical  perspective,  it  will  not  affect  the  model’s  predictions  of  discrimination 
thresholds. 


3.  HUMAN  PERFORMANCE  MODEL  SYSTEM  PARAMETERS  AND 

CALIBRATION 

The  Human  Performance  Model  was  calibrated  by  comparing  the  output  of  the 
model  with  psychophysical  data.  A  simplex  minimization  routine  (Press,  Flannery, 
Teukolsky  and  Vetterling,  1988)  was  used  to  change  the  parameters  of  the  model  in  order 
to  minimize  the  difference  between  the  output  of  the  model  and  the  psychophysical  data. 
In  what  follows  we  first  describe  the  detector  that  will  be  used  in  the  model.  We  then 
show  how  the  Human  Performance  Model  generates  threshold  data.  Finally  we  discuss 
the  psychophysical  data  that  was  used  to  calibrate  the  HPM  and  show  the  results  of  the 
calibration. 


3.1  Computing  Discrimination  Thresholds 


Psychopysical  data  are  often  reported  using  threshold  contrast  (the  contrast  required 
for  detection  or  discrimination)  or  contrast  sensitivity  (the  reciprocal  of  threshold 
contrast).  Because  we  wish  to  use  psychophysical  data  to  calibrate  the  Human 
Performance  Model  we  must  compute  HPM  System  contrast  thresholds.  The  equation 
used  to  compute  contrast  thresholds  uses  the  Quick- Wilson  formulation: 


D{c)= 


^j/?fej'r,(c)-  Rbase^{c^ 


m 
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where  Rtestj{c)  is  the  response  of  the  ith  cortical  mechanism  to  the  test  stimulus, 
RbaseXc)  is  the  response  of  the  ith  cortical  mechanism  to  the  base  stimulus,  and  D(c)  is 
a  scalar  representing  the  discriminability  of  the  two  stimuli.  From  the  prior  discussion  it 
follows  that  when  D(c)  =  1  then  C  is  equal  to  the  discrimination  contrast  threshold  of 
the  HPM  and  1/  C  is  the  discrimination  contrast  sensitivity  of  the  HPM.  To  find 
discrimination  contrast  thresholds  we  iteratively  changed  the  contrast  of  the  test  stimulus 
input  to  the  model  until  i2(c)  =  1  (within  a  0.01%  tolerance).  Note,  for  absolute 
thresholds  (in  which  an  observer  is  forced  to  discriminate  between  a  test  stimulus  and  a 
blank  screen),  Rbase^  (c)  =  0  for  all  i  so  the  differencing  operation  can  be  omitted. 

The  model  was  calibrated  by  comparing  the  thresholds  of  the  model  to  human 
psychophysical  data  and  adjusting  the  parameter  values  until  the  difference  between  the 
model  and  the  data  was  minimized.  The  error  for  a  given  set  of  stimuli  was  computed  as 

error  =  ^  |ln(Model  Threshold)  -  ln(Psycoptysical  Tlreshold)| 


and  a  simplex  minimization  routine  (Press,  Flannery,  Teukolsky  and  Vetterling,  1988) 
was  used  to  iterate  the  Human  Performance  Model  parameters  until  the  error  was 
minimized.  The  natural  logarithm  of  the  data  was  used  because  the  error  bars  for 
psychophysical  data  is  approximately  normalized  when  the  data  is  plotted  on  a 
logarithmic  scale. 


3.2  Luminance  Gain  Control 


3.2.1  Psycophysical  Data 

The  curves  shown  in  Figure  5  were  generated  using  an  equation  developed  by  Barten 
(1990).  The  equation  describes  the  effects  of  mean  luminance  and  image  size  on  human 
contrast  sensitivity  and  was  developed  from  experimental  data  collected  by  van  Meeterin 
(1973)  and  Carlson  (1982).  The  Barten  equation  was  used  to  calibrate  the  HPM.  It  has 
two  major  benefits.  First,  the  equation  describes  the  dependence  of  contrast  sensitivity 
functions  on  both  mean  luminance  and  stimulus  size.  Second,  the  data  were  collected 
vmder  conditions  that  correspond  to  those  of  a  pilot  in  a  cockpit,  i.e.,  observation  with 
both  eyes,  natural  pupils  and  prolonged  viewing  time. 

A  given  curve  in  Figure  5  shows  how  human  contrast  sensitivity  changes  with  spatial 
frequency.  The  top  curve  represents  sensitivities  measured  at  a  mean  luminance  of  1000 
candela/metei^.  For  each  successively  lower  curve  the  mean  luminance  was  reduced  by  an 
order  ofmagnitude  (a  factor  of  10).  The  majority  of  display  monitors  have  mean 
luminances  in  the  range  of  1  to  500  candela/meter^. 
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Figure  5.  Effects  of  mean  luminance  on  contrast  sensitivity. 


Inspection  of  Figure  5  shows  that  contrast  sensitivity  increases  with  mean 
luminance.  These  luminance-dependent  gain  changes  are  quite  large  particularly  at  high 
spatial  frequencies.  For  example  at  1  cycle/degree  sensitivity  changes  by  about  a  factor  of 
about  2.5  over  the  full  range  of  mean  luminances  and  at  10  cycles/degree  the  sensitivities 
change  by  about  a  factor  of  25. 

Those  familiar  with  contrast  sensitivity  functions  may  have  noticed  the  low 
frequency  falloff  of  the  curves  in  Figure  5  above  are  much  more  gradual  than  those 
typically  presented  in  vision  research  literature.  This  is  because  contrast  sensitivity 
functions  are  typically  measured  using  a  grating  stimulus  that  has  a  fixed  size.  Because 
the  spatial  frequency  bandwidth  of  a  sinewave  grating  stimulus  is  constant  for  all  spatial 
frequencies  when  grating  size  is  held  constant,  we  will  refer  to  these  more  typical 
measurements  as  constant  absolute  bandwidth  contrast  sensitivity  functions. 
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The  contrast  sensitivity  functions  in  Figure  5  result  when  the  width  of  the  sine  wave 
grating  changes  such  that 

Widthide^xSpatialFrequency {cycles  I  deg)  =  I5{cycles). 

We  will  refer  to  these  functions  as  constant  relative  bandwidth  contrast  sensitivity 
functions  because  the  stimulus  spatial  frequency  bandwidth  is  constant  when  measured  in 
octaves.  The  utility  of  using  constant  relative  bandwidth  stimuli  with  1.5  cycles  to 
calibrate  the  HPM  follows  from  the  fact  that  such  stimuli  are  consistent  with 
neurophysiological  data  and  are  similar  in  shape  to  the  linear  filters  of  the  HOP  transform. 
Because  of  the  close  match  between  the  stimuli  and  the  HOP  filters,  a  1.5  cycle  grating 
will  only  produce  robust  responses  in  a  few  cortical  mechanisms  and  pooling  (probability 
summation)  across  mechanisms  is  minimized.  Thus,  1.5  cycle  constant  relative 
bandwidth  contrast  sensitivity  functions  closely  approximate  the  luminance  gain  control 
required  by  the  HPM  and  are  particularly  useful  in  calibrating  the  luminance  gain  control 
parameters  of  the  HPM.  The  symbols  in  Figure  5  represent  the  model’s  fit  to  the  Barten 
functions  which  were  derived  from  empirical  data. 

3.2.2  System  Equation 


The  luminance  gain  control  module  was  implemented  in  the  model  as  a  luminance- 
dependent  two-dimensional  difference  of  Gaussians  (DOG)  linear  filter.  This  filter  is 
convolved  with  the  input  image  and  has  the  effect  of  attenuating  the  contrast  of  the  image 
in  a  frequency  dependent  manner.  The  form  of  the  function  is  as  follows: 


/(r)  =  M,,„,,,exp 


^2 


-M, 


surround 


exp 


^surround 


where  r  is  the  radial  distance  (in  degrees  of  visual  angle),  and  and 

G  are  parameters  to  be  fit  as  a  function  of  luminance.  The  function  is  a  very  good 

fit  to  the  spatial  weighting  function  of  neurons  in  the  retina  (Rodiack,  1965)  and  lateral 
geniculate  nucleus  (Irvin,  Casagrande  and  Norton,  1993;  Irvin,  Norton  and  Casagrande, 
1986). 


3.3  Probability  Summation  Across  Mechanisms 
3.3.1  Psychophysical  Data 

Figure  6  compares  a  constant  relative  bandwidth  contrast  sensitivity  function 
(bottom  curve)  and  a  constant  absolute  bandwidth  contrast  sensitivity  function  (top 
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ciirve).  Both  functions  were  measured  at  100  candela/meter^.  The  symbols  represent  the 
models  fit  to  the  data.  As  stated  above,  a  1.5  cycle  constant  relative  bandwidth  grating 
stimulus  will  only  produce  robust  responses  in  a  few  cortieal  mechanisms.  However,  a 
2.5  degree  constant  absolute  bandwidth  stimulus  will  stimulate  many  mechanisms.  In  the 
Human  Performance  Model,  the  increased  eontrast  sensitivity  of  the  constant  absolute 
bandwidth  curve  is  primarily  due  to  probability  summation  across  the  difference 
response  of  the  cortical  mechanisms.  The  HPM  equation  that  controls  pooling  across 
mechanisms  is  provided  below. 


Figure  6.  Probability  Summation  across  mechanisms. 

3.3.2  System  Equation 

There  are  two  parameters  that  affect  the  degree  of  summation  across  mechanisms. 
The  first  is  N,  the  number  of  mechanisms  which  contribute  to  the  summation  process, 
and  the  second  is  Q  which  determines  the  efficiency  of  the  summation  process.  These 
parameters  are  shown  in  the  Quick-Wilson  function. 
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D  = 


^_^Rtest^  —  Rbase^ 


1=1 


1/(2 


which  states  that  the  HPM  response  (D )  is  the  Qth  root  of  the  sum  of  the  absolute 
difference  responses  of  the  individual  mechanisms  to  the  test  and  base  stimuli,  raised  to 
the  Qth  power. 

Psychophysical  studies  have  shown  that  as  the  width  of  a  grating  stimulus  is 
increased  contrast  sensitivity  first  increases  and  then  stabilizes  at  about  15  cycles  of  the 
grating  (Robson  and  Graham,  1984).  Because  HOP  filters  have  about  1 .5  cycles  in  their 
spatial  support,  approximately  10  filters  (with  optimal  frequencies  equal  to  the  grating 
frequency)  are  required  to  span  the  width  of  this  stimulus  and  100  filters  will  be  required 
to  fully  cover  (considering  overlap  in  spatial  support)  a  15  cycle  by  15  cycle  grating. 
Therefore  N  in  the  Quick- Wilson  equation  was  set  equal  to  100. 


Examination  of  the  Quick-Wilson  function  shows  that  if  Q  =  \  summation  is  linear. 
If  we  hold  the  difference  responses  of  all  mechanism  pairs  constant  and  increase  Q ,  then 
the  HPM  response  decreases.  In  the  limit,  (  0  =  °°)  the  HPM  becomes  a  peak  detector, 
i.e.  the  HPM  response  will  be  determined  exclusively  by  the  maximum  difference 
response.  If  we  define  the  probability  summation  efficiency  as  the  HPM  response 
relative  to  the  HPM  response  when  summation  is  linear,  Q  =  \,  then  the  effect  of 
changing  Q  on  probability  summation  efficiency  for  100  mechanism  pairs  with  identical 
difference  responses  is  shown  in  Figure  7. 
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Figure  7.  The  Effect  of  Q  on  Probability  Summation  Efficiency. 

In  the  model  Q  changes  as  a  power  function  of  the  response  of  the  most  sensitive 
mechanism  in  the  Human  Performance  Model, 

where  is  the  value  of  the  maximiun  responding  mechanism  (either  Rtest  or  Rbase  ) 

and  Agand  B^are  parameters  whose  values  are  fit  by  simplex  iterative  minimization 

(Press,  Flannery,  Teukolsky  and  Vetterling,  1988).  In  the  HPM,  Q  changes  from  about 
2.5  at  contrasts  near  absolute  thresholds  to  about  10  at  high  contrasts  which  is  similar  to 
values  obtained  in  psychophysical  studies  (Wilson  1980,  Watson  1987).  Evidence  that 
Q  changes  as  a  power  function  of  contrast  is  provided  in  Wilson  (1980). 


3.4  Contrast  Gain  Control 


3.4.1  System  Equation 

Contrast  gain  control  was  implemented  in  the  model  using  the  following  equation. 
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•  Rj  is  the  response  if  the  cortical  mechanism, 

•  5,  is  the  squared  modulus  of  the  HOP  transform  corresponding  to 
mechanism, 

•  is  a  weighted  (w^)  sum  of  squared  moduli,  and 

j 

•  k  and  d  are  constant  parameters  to  be  fit. 

A  mechanistic  interpretation  of  the  equation  is  shown  below. 


Figure  8.  Diagram  of  Contrast  Gain  Control. 


The  cartoon-like  receptive-field  characterizations  on  the  left  hand  side  of  Figure  8 
represent  the  application  of  individual  HOP  gabor  filters  to  the  image  to  produce 
corresponding  HOP  transform  coefficients.  The  odd  and  even  symmetric  coefficients  are 
squared  (SQ)  and  summed  to  produce  a  squared  modulus.  A  weighted  summation  of  the 
squared  moduli  determines  the  strength  of  a  feedback  signal  that  divides  the  HOP 
coefficients.  This  model  has  recently  been  shown  to  be  consistent  with  the  data  obtained 
from  both  simple  and  complex  cells  in  the  primary  visual  cortex  (Heeger,  1991). 

Figure  9  shows  the  contrast  response  function  of  a  cortical  mechanism  at  low 
contrast  (0.0  -  0.2).  At  very  low  contrast,  where  the  feedback  signal  is  very  small,  the 
denominator  of  the  above  equation  is  approximately  1.0  and  the  response  is 
approximately  equal  to  the  squared  modulus.  As  contrast  is  increased,  the  feedback 
begins  to  contribute  to  the  denominator  and  the  response  begins  to  compress.  The 
compression  is  shown  more  clearly  in  Figure  10  in  which  the  response  is  shown  over  a 
contrast  range  of  0.0  -1.0. 


Figure  9.  Contrast  Response  Function  of  a  Cortical  Mechanism  with  and 
without  Contrast  Gain  Control. 
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3.4.2  Psychophysical  Data 

In  order  to  calibrate  the  contrast  response  fiinction  of  the  HPM  we  must  compare 
the  model  with  suprathreshold  psychophysical  data.  Figure  1 1  shows  increment 
threshold  data  provided  in  Legge  (1984).  In  this  study  the  subject  was  presented  with 
two  stimuli.  The  first  stimulus  consisted  of  a  sinewave  grating  of  fixed  contrast  which  we 
will  call  the  background  (Rbase).  The  second  stimulus  (Rtest)  was  a  sinewave  grating  that 
was  identical  to  the  first  grating  except  that  the  contrast  was  variable.  Legge  used  a  two- 
alternative  forced  choice  technique  (discussed  above)  to  determine  the  contrast  difference 
between  the  background  and  test  required  to  produce  a  75%  correct  response  (contrast 
threshold).  Legge  repeated  tiiis  procedure  using  a  variety  of  background  contrasts  and 
spatial  fi'equencies. 
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NORMALIZED  BACKGROUND  CONTRAST 

Figure  11.  Normalized  increment  threshold  data. 

There  are  three  prominent  features  of  this  data.  First,  at  low  contrasts  there  is  a  dip 
in  the  data  indicating  that  threshold  contrast  for  discrimination  is  less  than  absolute 
threshold  (horizontal  dotted  line).  Second,  as  the  background  contrast  is  increased  the 
threshold  contrast  begins  to  rise  and  cluster  near  the  straight  line,  which  in  this  figure  is  a 
power  function  with  an  exponent  of  0.5.  Finally,  the  data  from  all  spatial  frequencies 
(when  suitably  normalized)  cluster  around  a  single  template  given  by  the  solid  line  in  the 
figure. 

The  normalization  factor  used  to  generate  this  figure  was  the  absolute  threshold  of 
the  background  stimulus,  i.e., 

c 

^abs 

where  C  is  contrast  and  T  is  the  absolute  threshold  of  the  background  stimulus  and 
c^orm  normalized  contrast. 
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Of  course,  the  normalized  threshold  contrasts  (vertical  axis)  for  the  different  spatial 
frequency  gratings  must  equal  1  when  background  contrast  is  0  because  we  are  simply 
dividing  the  absolute  threshold  contrast  of  the  stimulus  by  itself  More  interesting  is  the 
finding  that  when  the  background  contrast  is  divided  by  the  normalization  factor  the 
minimum  of  the  dip  in  the  increment  threshold  data  for  all  frequencies  occurs  near  a 
normalized  background  contrast  of  1  and  that  the  threshold  contrasts  measured  at  higher 
background  contrasts  cluster  near  the  straight  line  in  Figure  1 1 .  The  straight  line  in  the 
figure  is  a  plot  of  the  following  power  function 

T{C^orm)  =  i<^normf^- 

This  experiment  provides  compelling  evidence  that  the  contrast  response  function  of 
the  visual  system  is  relatively  invariant  as  a  function  of  spatial  frequency. 

This  data  also  provides  evidence  about  the  shape  of  the  visual  system’s  contrast 
response  function.  If  the  contrast  response  function  of  the  HPM  was  linear,  then 
increment  thresholds  would  be  independent  of  background  contrast,  i.e.  the  data  would 
fall  on  the  dotted  line  labeled  absolute  threshold.  However  the  data  show  that  threshold 
contrasts  decrease  at  low  contrasts  and  increase  at  higher  contrast.  The  data  is  therefore 
consistent  with  the  model’s  contrast  response  function  shown  above  which  has  an 
accelerating  nonlinearity  at  low  contrasts  followed  by  a  compressive  nonlinearity  at  high 
contrasts  (Figures  9  and  10). 


3.4.3  Calibration  Results 

Legge’s  finding  has  been  replicated  and  extended  by  several  studies  (Bradley  and 
Ohzawa,  1986;  Ross  and  Speed,  1991).  All  of  the  studies  show  that  if  increment 
threshold  data  are  normalized  by  absolute  thresholds  then  the  data  cluster  near  a  canonical 
template.  However  the  intercept  and  exponent  of  the  power  function  describing 
threshold  contrasts  at  high  background  contrast  is  clearly  different  across  the  various 
studies.  We  fit  the  Human  Performance  Model  at  high  backgrounds  contrasts  with  a 
power  function  having  an  intercept  of  0.5  and  an  exponent  0.7, 

which  constitute  the  most  representative  values  across  the  various  studies. 

The  results  provided  by  this  calibration  are  shown  in  Figure  12  for  stimulus  spatial 
frequencies  of  1, 4,  and  16  cycles/degree.  These  data  are  plotted  as  Normalized  Threshold 
Contrast  values  to  facilitate  comparison  with  Legge’s  data  in  Figure  11. 


20 


Logic  Normalized  Background  Contrast 
Figure  12.  Normalized  increment  thresholds  of  model. 

4.  APPLICATION  OF  THE  HUMAN  PERFORMANCE  MODEL 

The  HPM  has  been  implemented  as  a  set  of  four  modules  that  can  be  linked  into  the 
user’s  application.  A  demonstration  program  included  on  a  distribution  diskette  serves  to 
illustrate  the  use  of  these  functions  in  a  complete  application  that  reads  in  a  pair  of  user- 
specified  images  and  prints  out  an  estimated  human  sensitivity  value.  An  abbreviated 
version  of  the  demonstration  program  appears  in  Listing  1  below.  The  functions 
InitModelParameters,  ApplyLgnFilter,  Apply CortexTransform,  and  ComputeSensitivity 
make  up  the  complete  interface  to  the  HPM  library  (see  steps  3,  4,  5  and  6  in  Listing  1). 
The  main  module  of  the  demonstration  program  also  calls  several  other  functions  which 
are  defined  in  the  complete  demonstration  software  source  listings.  These  functions  serve 
to  “read”  the  pair  of  images  and  their  associated  parameter  file. 
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/♦ 

DEMO  -  Abreviated  main  program. 

Usage;  demo  imagel.tif  image2.tif 

*/ 

#mclude  <stdio.h> 

#include  "../hpmlib/hpm.h" 

#mclude  "demo.h" 
main(int  argc,  char  *argv[]) 

{ 

IMAGE?  ARAMETER  ImageParam; 

LGNPARAMETER  LgnParam; 

CORTEXPARAMETER  CortexParam; 

DISCRIMPARAMETER  DiscrimParam; 

ROI  RegionOflnterest; 

MULTICHROME  *plmagel; 

MULTICHROME  ♦plmage2; 

MULTICHROME  *pLgnI; 

MULTICHROME  *pLgn2; 

MULTICHROME  *pCortexl; 

MULTICHROME  *pCortex2; 
float  fMeanLuml; 
float  fMeanLum2; 
float  fLuminance; 
float  fSensitivity; 

// 

//  (1)  READ  IMAGE  PARAMETERS  FROM  INITIALIZATION  FILE 
// 

ReadlmageParametersC'image.ini",  &ImageParam); 

// 

//  (2)  READ  AND  RESCALE  IMAGE  PAIR  &  COMPUTE  MEAN  LUMINANCES 
// 

pimagel  =  Readlmage(argv[l],  &ImageParam,  &fMeanLuml); 
plmage2  =  Readlmage(argv[2],  &ImageParam,  &fMeanLum2); 

//  (3)  INITIALIZE  PARAMETERS  USING  SPECIFIED  BACKGROUND  LUMINANCE 
// 

fLuminance  =  max(fMeanLuml,  fMeanLum2); 

InitModelParametersC&LgnParam,  &CortexParam,  &DiscrimParam,  fLuminance); 

//  (4)  APPLY  LGN  FILTER  TO  RECT  IMAGES  TO  OBTAIN  FILTERED  HEX  IMAGES 
// 

pLgnl  =  ApplyLgnFilterfpImagel,  &ImageParam,  &LgnParam); 
pLgn2  =  ApplyLgnFilter(pImage2,  &ImageParam,  &LgnParam); 

// 

//  (5)  APPLY  CORTEX  TRANSFORM  TO  PAIR  OF  FILTERED  HEX  IMAGES 
// 

GetROIForReferenceImage("image.ini",&RegionOfInterest); 

pCortexl  =  ApplyCortexTransformfpLgnl,  &RegionOfInterest,  &CortexParam); 

GetROIForTestImageC'image.ini",  &RegionOf[nterest); 

pCortex2  =  Apply CortexTransform(pLgn2,  &RegionOfInterest,  &CortexParam); 

//  (6)  COMPUTE  DISCRIMINABILITY  OF  PAIR  OF  CORTEX  REPRESENTATIONS 
// 

fSensitivity  =  ComputeSensitivity(pCortexl,  pCortex2,  &DiscrimParam); 
printfC'Discrimination  sensitivity  =  %f\n",  fSensitivity); 

} _ _ _ 
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The  demonstration  program  illustrated  above  represents  the  most  basic  application 
of  the  HPM  to  implement  a  system  that  predicts  human  performance  in  a  discrimination 
task.  The  demonstration  program  can  also  predict  human  performance  in  a  simple 
detection  task  (i.e.,  target  on  uniform  background)  if  one  specifies  one  of  the  two  input 
images  as  a  uniform  luminance  field. 

The  Human  Performance  Model  has  been  designed  to  provide  the  basis  for  the 
development  of  more  sophisticated  applications.  In  particular,  the  HPM  has  been 
designed  to  facilitate  the  construction  of  diverse  systems  that  act  as  surrogate  human 
observers.  A  surrogate  human  observer  is  here  defined  as  any  automated  system  used  to 
predict  human  visual  psychometric  functions  without  having  to  conduct  psychophysical 
experiments  with  real  human  subjects. 

For  example,  there  are  many  situations  where  one  would  like  to  know  how  human 
discrimination  sensitivity  varies  as  a  function  of  an  independent  variable.  In  particular, 
one  might  be  given  target  and  decoy  infi-ared  (IR)  images  collected  at  regular  intervals 
throughout  the  day.  Taking  these  image  pairs  as  input,  a  simple  application  could  use  the 
HPM  to  produce  a  plot  of  predicted  human  target  vs.  decoy  discrimination  sensitivity  as 
a  function  of  time-of-day.  Note,  that  the  target  and  decoy  objects  should  be  presented 
against  equal  backgrounds,  be  depicted  from  equal  viewpoints,  and  hence  differences  in 
their  images  should  reflect  only  intrinsic  differences  in  shape  and  thermodynamic 
properties. 

There  are  also  many  situations  where  one  desires  the  accurate  prediction  of  human 
visual  performance  in  more  complicated  tasks,  such  as  detecting  a  target  on  a  cluttered 
background,  target  recognition,  and  target  identification.  Fortunately,  human  visual 
performance  tasks  such  as  these  can  often  be  formulated  in  terms  of  paired  image 
discriminations.  After  so  reformulating  the  problem,  one  can  then  proceed  to  develop  an 
application  that  uses  the  HPM  to  predict  human  performance  in  these  more  complicated 
tasks. 
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